1. Field of the Invention
The present invention relates to an image processing method and an image processing apparatus, and more particularly to an image processing method and an image processing apparatus in which a distance image obtained by allocating distance information to each pixel is masked with a key signal generated from a processing target image, thereby clearly displaying the outline of a synthesized image based to the distance image.
2. Description of the Related Art
It has been hitherto adopted in a broadcasting studio, etc., that a picture obtained by a live camera, a picture based on computer graphics, etc. are superimposed on one another by using a chromakey method, and the composite image thus formed is used for broadcasting or the like. That is, according to the image composite technique based on the chromakey method, a picture of an object such as a person or the like who stands in front of a predetermined screen is picked up by a television camera. The screen is designed to have a color such as blue, green or the like which excludes colors contained in the picture of the object. A key signal is generated on the basis of the color of the screen from the picture signal from the television camera, and the picture signal based on the computer graphics and the picture signal from the television camera are subjected to keying processing on the basis of the key signal.
Accordingly, according to the image composite technique based on the chromakey method, various pictures can be generated as if a person exists in a virtual space based on computer graphics, at a remote place or the like.
In the conventional chromakey processing, the pictures are superimposed on the basis of the key signal based on the color of the background without taking the distance of the object from the television camera into consideration, and thus this technique has a problem that it is difficult to obtain such a composite image having a far-and-near background as if a person who appears from the back of an object based on computer graphics moves to the front side of the object.
A method of solving this problem may be a method of superimposing pictures on the basis of a keying based on distance information in place of the keying based on the color. That is, for every picture to be superimposed, information on the distance to an imaging device is successively set to each pixel to generate a picture (hereinafter referred to as "distance image"). Here, when the distance image is observed through a monitor, the distance from each pixel to the object is represented by light and darkness of the pixel.
In the case where the object is based on the computer graphics, the distance image can be generated by calculating the distance between the viewing point and the object for each pixel and successively allocating the distance information thus obtained to each pixel. On the other hand, in the case where the object is taken on the spot, the distance image can be generated by stereoscopically viewing the object from a plurality of imaging devices and detecting the parallax of every pixel between the pictures obtained by the a plurality of imaging devices.
In the image composite technique using the keying based on the distance information, the corresponding distance information is successively compared between these two distance images (the distance image of the picture taken on the spot and the picture based on the computer graphics) to successively select image data at the viewing point side from the two processing target pictures. This processing is executed on continuous pictures to generate such a picture sequence that a person on the spot which appears from the back of an object moves to the front side of the object.
The distance image based on the picture taken on the spot is actually generated as follows: a predetermined area comprising a plurality of pixels is successively displaced on an image serving as a comparison target to detect a position at which the images picked by the imaging devices are coincident with each other, the displacement of the predetermined area between these images is detected, and the displacement of this area thus detected is set as the parallax of the center pixel in this area. Accordingly, it is difficult to accurately calculate the distance at the edge portion of the subject and at the background side in the distance image which is generated for every predetermined area comprising a plurality of pixels as described above. Therefore, the outline of the composite image is displayed unclearly, and the image quality of the composite image is lowered.